Method, apparatus, device and storage medium for implementing augmented reality scene

ABSTRACT

A method, an apparatus, a device, and a storage medium for implementing an augmented reality scene. The method including obtaining movement sensing data of a target object acquired by a positioning apparatus, determining, according to the movement sensing data, space motion information of the target object in the target site area and updating, according to the space motion information, an object model of the target object in a three-dimensional scene model corresponding to the target site area, determining an object position of the updated object model in a target area, the target area being an area determined in the three-dimensional scene model according to position information and field of view information of the display device in the target site area, and displaying virtual information on the display device according to the object position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2018/114177 filed on Nov. 6, 2018, which claims priority fromChinese Patent Application No. 201711216327.2, filed in the ChinesePatent Office on Nov. 28, 2017, and entitled “METHOD, APPARATUS, DEVICEAND STORAGE MEDIUM FOR IMPLEMENTING AUGMENTED REALITY SCENE”, which areincorporated herein by reference in their entireties.

BACKGROUND 1. Field

Embodiments of the present disclosure relate to the field of augmentedreality technologies, and in particular, to a method, an apparatus, adevice and a storage medium for implementing an augmented reality scene.

2. Description of Related Art

Augmented Reality (AR) is a technology that combines relevantinformation of the real world and virtual information to build anaugmented reality scene and represents the augmented reality scene tousers. The AR technology may superimpose, based on a specific spatialarea in the real world, a real scene and virtual information into a samespace image to be sensed by sense organs of AR users, therebyimplementing sensory experience beyond reality.

SUMMARY

Embodiments of the present disclosure provide a method, an apparatus, adevice and a storage medium for implementing an augmented reality scene,and may generate virtual information in the augmented reality scenebased on sensing movement of an object.

According to an embodiment, there is provided a method for implementingan augmented reality scene, executed by a computing device, the methodincluding obtaining movement sensing data of a target object acquired bya positioning apparatus, the positioning apparatus being configured tomonitor the target object in a target site area; determining, based onthe movement sensing data, space motion information of the target objectin the target site area; updating, based on the space motioninformation, an object model of the target object in a three-dimensionalscene model corresponding to the target site area to obtain a firstupdated object model of the target object; and determining an objectposition of the first updated object model in a target area to display,on a display device, virtual information according to the objectposition, the target area being an area determined in thethree-dimensional scene model based on position information and field ofview information of the display device in the target site area.

According to another embodiment, there is provided a method forimplementing an augmented reality scene, executed by a display device,the method including obtaining position information and field of viewinformation of the display device in a target site area, the displaydevice being configured to display virtual information of the augmentedreality scene; generating orientation indication information based onthe position information and the field of view information, andtransmitting the orientation indication information to an augmentedreality processing device, the orientation indication informationinstructing the augmented reality processing device to generate thevirtual information to be displayed in a user field of view determinedby the position information and the field of view information; andreceiving a display message carrying the virtual information, from theaugmented reality processing device, and displaying the virtualinformation based on the display message, the display messageinstructing the display device to overlay-display the virtualinformation in the user field of view.

According to another embodiment, there is provided an augmented realitydisplay device, including at least one memory configured to storecomputer program code; and at least one processor configured to accessthe computer program code and operate as instructed by the computerprogram code, the computer program code including obtaining codeconfigured to cause the at least one processor to obtain movementsensing data of a target object acquired by a positioning apparatus, thepositioning apparatus being configured to monitor the target object in atarget site area; model update code configured to cause the at least oneprocessor to determine, based on the movement sensing data, space motioninformation of the target object in the target site area, and update,based on the space motion information, an object model of the targetobject in a three-dimensional scene model corresponding to the targetsite area to obtain a first updated object model of the target object;determining code configured to cause the at least one processor todetermine an object position of the first updated object model in atarget area, the target area being an area determined in thethree-dimensional scene model according to position information andfield of view information of the display device in the target site area;and display code configured to cause the at least one processor todisplay virtual information on the augmented reality display deviceaccording to the object position.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of the presentdisclosure more clearly, the following are described with reference tothe accompanying drawings. Apparently, the accompanying drawings in thefollowing description show merely some embodiments of the presentdisclosure, and a person of ordinary skill in the art may still deriveother embodiments from these accompanying drawings without creativeefforts.

FIG. 1a is a schematic diagram of a scene construction according to anembodiment.

FIG. 1b is a schematic diagram of a sensing method of movement sensingdata according to an embodiment.

FIG. 1c is another schematic diagram of a sensing method of movementsensing data according to an embodiment.

FIG. 1d is still another schematic diagram of a sensing method ofmovement sensing data according to an embodiment.

FIG. 2 is a schematic diagram of AR glasses according to an embodiment.

FIG. 3a is a schematic diagram of a site content seen through a pair ofAR glasses according to an embodiment.

FIG. 3b is another schematic diagram of a site content seen through apair of AR glasses according to an embodiment.

FIG. 3c is still another schematic diagram of a site content seenthrough a pair of AR glasses according to an embodiment.

FIG. 4 is a schematic structural diagram of an implementing system of anaugmented reality scene according to an embodiment.

FIG. 5 is a flowchart of a method for implementing an augmented realityscene according to an embodiment.

FIG. 6 is a flowchart of a position information determining method basedon distance data according to an embodiment.

FIG. 7 is a flowchart of a virtual information updating method accordingto an embodiment.

FIG. 8 is a flowchart of another method for implementing an augmentedreality scene according to an embodiment.

FIG. 9 is a schematic structural diagram of an apparatus forimplementing an augmented reality scene according to an embodiment.

FIG. 10 is a schematic structural diagram of another apparatus forimplementing an augmented reality scene according to an embodiment.

FIG. 11 is a schematic structural diagram of an augmented realityprocessing device according to an embodiment.

FIG. 12 is a schematic structural diagram of an augmented realitydisplay device according to an embodiment.

FIG. 13 is a schematic structural diagram of a system for implementingan augmented reality scene according to an embodiment.

DESCRIPTION OF EMBODIMENTS

In the current implementation of the AR technology, an environment imagemay be captured by using a camera. A position in which virtualinformation is required to be overlaid may be recognized in the imageand virtual information may be generated and overlaid in the capturedimage. An AR processing device is required to have good instant imagerecognition performance and is also required to recognize an imagecurrently captured by the camera in real-time. As such, high-endsoftware and hardware resources are consumed to complete AR processing.

According to embodiments of the present disclosure, one or more sceneareas may be predisposed. A various scene props may be disposed in eacharea. For example, when a park scene area is disposed, objects such asflower stands, sculptures of various shapes, tables, and chairs may bedisposed in the park scene area. In the disposed area, based on a shapeof each object in an actual area in the real world and a position in thescene thereof, a virtual three-dimensional scene model corresponding tothe area may be generated, and the virtual three-dimensional scene modelmay be used to completely represent shapes and positions of objects inan entire area.

An AR user may use a display device to move in the area in the realword. The display device may be a pair of AR glasses with transparentdisplay screens. On one hand, the user may directly observe some areasin the area in the real word by using the AR glasses. On the other hand,the AR glasses may display virtual information which needs to bedisplayed. As such, the user may view the virtual information whenobserving the area. For example, when actually viewing a tree in thearea by using the AR glasses, the user may also view some overlaidvirtual information about the tree, and the virtual information may beinformation such as a name, an attribute, and a source of the tree. Inthe moving process of the user wearing the AR glasses, when a positionof the AR glasses in the area scene and a field of view of the ARglasses are known, a user field of view in this case may be calculated.When the position of the AR glasses in the area scene, and thecalculated view range of the user are combined with thethree-dimensional scene model, a particular range of view may bedetermined so that the user wearing the AR processing device may view anobject in a particular range of view, and then virtual informationrequired to be overlaid may be generated. The generated virtualinformation and a three-dimensional space model of the object may becalculated to determine a position where the virtual information shouldbe displayed. The generated virtual information may be virtualinformation that should be displayed and that is found, or may bevirtual information that is obtained after being further processed basedon virtual information that has been found and that has been displayed.

According to an embodiment, when the virtual information is informationsuch as text that can be displayed directly on the AR glasses, a displayposition of the virtual information may be calculated based on an objectmodel corresponding to an object in the view range of the user, and theAR glasses directly displays the virtual information on the calculateddisplay position. For example, if the virtual information about the treeneeds to be displayed on a trunk, and a display position of the virtualinformation on the AR glasses needs to be calculated based on a positionof the trunk in an object model of the tree in a view range of the user,the AR glasses displays the virtual information at the position of thetrunk.

Furthermore, if the virtual information is a virtual image, it needs tobe determined whether the virtual image is blocked by an object in thearea in the view range of the user. Based on the object modelcorresponding to the object in the view range of the user, the displayposition where the virtual image should be displayed, and a shape of thevirtual image, it needs to determine the area in which the virtual imagemay be displayed. For example, the virtual image may be displayed insome areas and may not be displayed in other areas. The AR glasses maydisplay the virtual image in only some areas. For example, a virtualstreet lamp may need to be displayed to the user. In this case, someareas that are blocked by a desk and a tree in the virtual street lampmay need to be removed based on object models corresponding to the deskand the tree in the view range of the user, and a street lamp imagecorresponding to some unblocked areas is obtained and displayed.

According to an embodiment, the AR glasses may only display virtualinformation. A dedicated AR processing device, which may be a computingdevice such as a server, may be configured to calculate athree-dimensional scene model in the site area. Based on a field of viewof a position of the AR glasses, a user field of view and processing ofthe virtual information may be calculated. The server may transmit thegenerated virtual information carried in a display message to the ARglasses, and the AR glasses may display the virtual information on acorresponding display position based on the virtual information and anindication of the display position in the display message. In this way,in the user field of view, the virtual information may be seen in acorresponding position. For example, virtual information of the tree maybe seen on the trunk of the tree, and the virtual image of a streetlight behind the table may be seen. Further, there may be a plurality ofsite areas. In such case, the site area where the AR user is currentlylocated may be determined as a target scene area based on a position ofthe AR glasses.

FIG. 1a is a schematic diagram of a scene construction according to anembodiment. The AR glasses 104 may provide an augmented reality serviceto the user only in an interactive range of a target site area 100 wherethe user is currently located, and an interaction between the user andAR may be effective only in the interactive range of the target sitearea 100. A distance sensor 102 in the target site area 100 may bepreset. There may be a plurality of the distance sensors 102, and thesedistance sensors 102 may locate a fixed object 106, a moving object anda person in the target site area 100 by means of echo, reflected light,or radio signal ranging in combination with positions of the distancesensor 102 and a matching receiver thereof. A specific positionalgorithm may be a triangulation position algorithm. A ground assistedpositioning device 103 may also disposed in the target site area 100.The ground assisted positioning device 103 may scan and detect an areaat a particular distance from the ground by using a laser array, toprocess objects and details which may be blocked. The ground assistedpositioning device 103 is mainly configured to avoid a problem ofinaccurate positioning of an object caused by that the distance sensor102 in which the distance sensor 102 cannot accurately sense theposition data of the object that is close to a ground area or some areasof the object, for example, some areas under a stool bottom of a stool.The AR glasses 104 is a wearable display. The AR glasses 104 mayinteract with an AR processing device 107 by radio signals, transmit animage required to be displayed, and transmit the position and a rotatingangle of the AR glasses 104, so as to determine position information andfield of view information of the display device in the target site area100.

The target site area 100 may include a target object 105. The targetobject 105 may be any moving object in the target site area 100, forexample, a movable chair and the like. The position of the target object105 may be determined by the distance sensor 102, and a positioningsensor such as an acceleration sensor may be built in the object toimprove the accuracy of calculating the position of such object. Aplurality of positioning sensors such as acceleration sensors may bedisposed on different areas of the target object 105, so as toaccurately locate a position of the entire target object 105 in thetarget site area 100. The positioning data may be transmitted to the ARprocessing device 107 by radio. A position and a shape of a fixed object106 in the target site area 100 may be determined in advance. A detailedposition and a three-dimensional model of the fixed object 106 may bepre-recorded in a stored scene database, to ensure that the server mayeffectively construct the same interactive space as the real physicalspace when interacting with the user.

Furthermore, in the target site area 100, the fixed object 106 or amoving object may be a rigid object, and the positioning device maymonitor the entire object. FIG. 1b is a schematic diagram showing thatthe target object 105 is monitored as a rigid object to obtain movementsensing data. It is assumed that parts of a chair as the target object105 are fixedly connected to each other, and components do not moverelative to each other. The structure and shape of the target object 105as a rigid object do not change with the movement of the object. Thepositioning device may monitor a position and/or a rotating angle of onepoint in a moving object, to obtain the movement sensing data of theentire object, facilitating determining the position and the rotatingangle of the target object 105 based on the detected movement sensingdata. The position device may update the position of the target object105 through an overall movement and/or a rotation of the object modelcorresponding to the moving object.

In contrast, the moving object in the target site area 100 may beflexible. In this case, the sensing data of the moving object may bedetermined in two manners. First, a positioning sensor may be disposedin a key part of the target object 105, for example, the positioningsensor such as an acceleration sensor, or a gyroscope may be disposed.Movement sensing data such as the position and the rotating angle in thecorresponding area of the key part may be accurately obtained by usingthese positioning sensors disposed in the key part. As shown in FIG. 1c, it is assumed that the chair as the target object 105 is a chair witha seat surface 1501 that may be raised and lowered. As such, anacceleration sensor 1053 may be disposed on the seat surface 1051 and anacceleration sensor 1054 may be disposed on a back 1052. When the targetobject 105 is horizontally moved while raising the seating surface 1051based on the distance d, the two sensors can respectively sense the twomovement sensing data. Subsequently, based on the two movement sensingdata, the spatial position of the entire target object 105 after beingmoved and transformed may be obtained. However, FIG. 1c is only anexample.

Second, movement sensing data such as movement position of each part ofthe entire target object 105 after movement may be directly scanned byusing ranging sensor of array type such as a laser ranging array, aninfrared ranging array or an ultrasonic ranging array. As shown in FIG.1 d, the chair as the target object 105 is a transformable object. Forexample, the chair has a plurality of components that may move betweeneach other. In FIG. 1 d, the distance sensor 102 such as the laserranging array may obtain movement sensing data for the target object 105in an array scan manner. Subsequently, based on the movement sensingdata and the triangulation positioning algorithm, positions of aplurality of position points of the target object 105 in the target sitearea 100 may be determined. In FIG. 1 d, to ensure that some parts suchas the bottom part, which is not easily scanned by the distance sensor102, the ground assisted positioning device 103 may also obtain themovement sensing data. The data of the distance sensor 102 issupplemented to more accurately obtain the movement sensing data of mostposition points on the movable target object 105, facilitating thesubsequent update of the object model of the entire target object 105.It should be noted that, the arrays emitted by the distance sensor 102and the ground assisted positioning device 103 in FIG. 1d are used as anexample. In an actual case, based on the distance sensor 102 and theground assisted positioning device 103 that have different performancesand different quantities, more laser beams or ultrasonic waves may beemitted to complete the scanning of the entire target object 105.

According to an embodiment, in order to solve the problem of incompletescanning caused by the target object 105 being blocked by other objectsduring the movement, the environment image captured by the camera mayfurther be combined to recognize the image object of the target object105 in the image of the target site area, in a simple image recognitionmanner, to determine whether the target object 105 is blocked, and todetermine the position area which cannot be scanned by the distancesensor 102 and the ground assisted positioning device 103. Based on thedetermined position area which cannot be scanned, and with reference tothe data sensed by the distance sensor 102 and the ground assistedpositioning device 103, a part of the data may be filtered out from thesensed data for processing, which is used as the movement sensing dataof the target object 105 for subsequent processing.

The AR processing device 107 may be a computing device, such as a serveror a host, which may be configured to receive the sensed data of thedistance sensor 102 and the positioning sensor, process real-timepositioning information of the target object 105 and an object model ofthe target object 105 in a real scene, process interactive relationshipbetween virtual objects, generate virtual information 108 required to beoverlaid in the view range of the user, and transmit the generatedvirtual information 108 to the AR glasses 104 of the AR glasses 104 ofthe user. Furthermore, there may be a plurality of AR glasses 104 in thetarget site area 100. The AR processing device 107 may generate thevirtual information for each AR glasses 104, and transmit thecorresponding virtual information to each of the AR glasses 104 ofusers. An antenna configured for wireless communication may be disposedbased on common wireless Internet standard, so that the distance sensor102, the ground assisted positioning device 103 and the positioningsensor disposed on the target object 105 may transmit the movementsensing data of each target object 105 to the AR processing device 107.The AR processing device 107 may also transmit the AR scene processingresult, for example, display information, to the AR glasses 104. Themovement sensing data may be original data sensed by each sensor. Thatis, information, such as the position and the rotating angle of thetarget object obtained directly by each sensor, or data such as theposition and the rotating angle obtained after the data from each sensoris processed, may be calculated by the server. The server may directlyobtain the information such as the position and rotating angle of thetarget object.

The dashed line in FIG. 1a shows virtual information 108 that the usercan view through the AR glasses 104. Using the AR glasses 104, the usermay also view the stools and squares in the scene. According to anembodiment, some interactive elements may be added. For example, theuser may go under a virtual tree, look at the type of the virtual tree,and select annual rings of the virtual tree to view age information.Because coverage ranges of the field of view determined based on theposition information and the field of view information of the AR glasses104 are different, the view range of the user at the left side of inFIG. 1a may include a virtual tree and a virtual car, while the viewrange of the user at the right side may only include the virtual tree.The virtual information 108 overlaid by different display screens of theAR glasses 104 of the user may be determined based on the view range ofthe user. The calculation of the virtual information 108 that needs tobe displayed to the user may be completed by the AR processing device107.

The AR glasses 104 are a wearable display device. The AR glasses mayalso be another display device, such as a head up display (Head UpDisplay, HUD). The basic principle is to cover the view range of theuser by using a transparent display screen, and to sense positioninformation and rotating angle information of the AR glasses 104 byusing the positioning sensor and the angle sensor disposed on the ARglasses 104, to obtain the user's position information and field of viewinformation. The position information and the field of view informationare transmitted to the AR processing device 107. The AR glasses 104 arefurther configured to display the virtual information 108 existing inthe virtual space corresponding to the view range of the user returnedby the AR processing device 107, creating an illusion that the virtualinformation 108 exists in the field of view for the user. The ARprocessing device 107 may obtain and calculate a current position of theuser in the target site area 100 and a field of view rotating angle of ahead of the user relative to the initial position after receiving theposition information and the field of view information to further obtainthe user field of view, and determines the virtual information 108.

FIG. 2 is a schematic structural diagram of AR glasses 104 according toan embodiment. The AR glasses 104 may include a fixed device 1041, asensor component 1042 and a transparent display screen 1043, and mayfurther include structures such as a wireless component and a battery.The fixed device 1041 mainly ensures that the AR glasses 104 may befixed onto a head of the user. A display range of the AR glasses 104 mayeffectively cover most areas in the middle of the user field of view,and the sensor component 1042 may sense in real-time the position andangle information of the user field of view with the movement of theuser's head. The fixed device 1041 may use a simplest ear-mountedglasses frame solution, and may also use an elastic bandage or a fullcoverage helmet or headgear solution, or neck fixation solution. Thesensor component 1042 may be a component based on a six-axis gyroscopechip and/or an acceleration sensor chip, which may directly obtain theposition of the user in the target site area 100 and the rotating angleof the user's head. The sensor component 1042 may further include anauxiliary positioning receiver based on a laser, a wireless or an echo.The auxiliary positioning receiver uses a received signal (a laser, awireless or a sound wave) as an auxiliary. Based on a time difference ofthe laser, the wireless or the echo received by the auxiliarypositioning receiver, or a scan frequency difference, displacementsensing data of the AR glasses 104 may be obtained to determine theposition information and the field of view information of the AR glasses104. The wireless component and the wireless communication antenna inthe AR glasses 104 transmit and receive signals. The sensing data of theAR glasses 104 may be uploaded to the AR processing device 107, and theimage required to be displayed may be downloaded by the AR glasses 104.

According to an embodiment, the positioning sensor disposed on thetarget object 105 may include a combination of a plurality of sensorsand communication components. The communication component may be a wireconnected component and/or a wireless connected component of the ARprocessing device 107. Combined with a particular fixed device, thepositioning sensor may be fixed to the target object 105 in the targetsite area 100. The position and the rotating angle of the correspondingtarget object 105 may be recorded and transmitted in real-time. Apositioning sensor may be disposed at a plurality of key positions ofthe target object 105, for example, for the user in the target site area100, and may be made into a wearable positioning sensor, such as awristband or a back strap, to accurately determine a plurality of partsof the user's human body in the three-dimensional space in the targetsite area 100, thereby enhancing the augmented reality scene andimproving the user interaction experience.

In the AR scene, an interaction between the user and the object and aninteraction between the user and the preset system interface may also beimplemented. Specifically, a user operation may be received by disposinga touch pad or a button on an outer frame of a mobile display screen.Alternatively, by using the positioning sensor fixed on the user's hand,when the user makes some particular gestures, for example, moving afinger, a corresponding gesture response operation may be selected basedon pose recognition. For example, by using the positioning sensor fixedon the user's wrist, if a swing gesture is detected, the displayedinformation about the tree may be turned to the next page. If the touchpad or the button is used, data input by the user may be required to betransmitted back to the AR processing device 107. Upon receiving thedata, the AR processing device 107 determines how to handle the user'soperations on the touchpad or button. After the response processing, theprocessing result may be displayed on the AR glasses 104 in an imagemode.

From a perspective of a user, a real object in the target site area 100and the virtual information 108 may co-exist in the view range of theuser. With the movement of the view range, all the virtual objectsspatially change with the real objects to complete the display of theaugmented reality scene and implement the augmented reality scene. Asshown in FIG. 3 a, when the AR glasses 104 is used only as glasses, theuser can see the real object in the target site area 100 through theframe 301 where the transparent display screen 1043 is located, and thevirtual information 108 is not displayed. FIG. 3b shows the virtualinformation 108, that is, two virtual trees in FIG. 3 b. In FIG. 3 c,the user turns the head at a small angle, a new field of view isdetermined based on rotating angle, and two virtual trees are overlaidat a new position. The two virtual trees overlaid in FIG. 3c and the twovirtual trees shown in FIG. 3b occupy different areas on the displayscreen, taken from the two same virtual trees based on different fieldsof view determined by the AR glasses 104.

FIG. 4 is a schematic structural diagram of an implementing system of anaugmented reality scene according to an embodiment. As shown in FIG. 4,the system may include a server, a client and a data acquisitionterminal. The server may include one or more servers, and the server maycorrespond to the AR processing device 107. The client may be a displaydevice 402 that may be worn or carried by the user, for example, the ARglasses 104. The client may include the display devices 402 that may beworn or carried by a plurality of users experiencing AR scenes in thetarget site area. In FIG. 4, there is a plurality of display devices402, and a specific quantity of the display devices 402 may becomprehensively determined based on a size of the target site area and acomputing power of the AR processing server. The data acquisitionterminal may include the plurality of positioning devices 401. Thesepositioning devices 401 may include a sensor that senses the positionand rotating angle of a target object during movement, for example, adistance sensor, an acceleration sensor, an angle sensor, a groundassisted positioning device that are disposed on the target object andthe AR glasses, and may further include a vision sensor such as a camerafor capturing a target object for assisted positioning based on imagerecognition. A specific quantity of positioning devices 401 may beconfigured based on an update accuracy requirement of the object modelcorresponding to the moving object. For a target site area, amulti-directional, multi-angle, multi-number positioning device 401 maybe disposed to better detect position of a moving object. A signaltransmission processing device 403 may be a wireless transmission accesspoint device such as a WiFi device.

In an embodiment of this application, the AR processing device or thefunction of service may be implemented by using seven function modules,including: a sensor signal processing module 404, a position andorientation computing module 405, a view range computing module 406, avirtual object database 407, a physical effect processing module 408, animage rendering module 409, and an interactive control module 410.

The server may collect, by using the signal transmission processingdevice 403, movement sensing data such as the position and the angle ofthe target object sensed by the positioning device 401. Various kinds ofpositioning devices 401 collect the movement sensing data obtained byeach corresponding sensor at a time interval T_(p), the movement sensingdata received at t moment is recorded as P^(t)=(pos^(t), angle^(t)),where pos^(t)={pos₁ ^(t), . . . , pos₂ ^(t)}, pos^(t) represents themovement sensing data of the target object sensed by each positioningdevice 401 at t moment in the target site area. Sensors, such as a laserand an echo, generally include time or phase differences in signaltransmission of corresponding laser signals and echo signals. Theacceleration sensor may include the acceleration data, and angle^(t) isthe rotation angle data of the target object recorded by an anglesensor, such as a gyroscope included in the positioning device 401. Themoving data and the rotating angle may be used to calculate space motioninformation of the target object. Each positioning device 401 transmitsthe movement sensing data sensed at certain time interval to the signaltransmission processing device 403. Further, a particular degree ofnoise reduction and compression may be performed on the movement sensingdata P^(t). Noise data included in the sensing data of each positioningdevice 401 may be removed to obtain more accurate movement sensing data,and the movement sensing data may be compressed to reduce requirementsfor wireless transmission bandwidth. The noise reduction processing maybe performed by using Fourier transform noise reduction and wavelettransform noise reduction algorithms.

The display device 402, such as the AR glasses, parses and displays thereceived data to be displayed on the display screen. The data to bedisplayed mainly includes a virtual image to be displayed, virtualinformation, a display position on the screen of the display device 402,and the like. Because the data transmitted by the signal transmissionprocessing device 403 may be delayed or congested, a frame rate of dataneeds to be smoothed and skipped at different moments, and a differenceprediction may be performed on the image content when necessary.

The signal transmission processing device 403 is configured to performeffective data transmission, and complete data exchange between theserver, the client and the data acquisition terminal. A hardwareimplementation of the signal transmission processing device 403 mayemploy near field communication or a wireless local area networksolution. The signal transmission processing device 403 may match thecommunication interface disposed on the server, the client, and the dataacquisition terminal, and may be configured to implement the dataexchange between the server, the client, and the data acquisitionterminal by using these interface transmission data. For example, thesignal transmission processing device 403 may communicate with asubmodule having a transmitting and receiving function disposed on thedisplay device 402, and the signal transmission processing device 403may communicate with a transmitting submodule disposed on thepositioning device 401. Further, the interface for data transmission onthe server may include functions, such as routing and link management inaddition to sending and receiving functions.

The signal transmission processing device 403 may be controlled by meansof multi-line management in software, and the data to be exchanged maybe queued and processed by using a queuing processing algorithm. Forsignal transmission, a compression algorithm may be used for encoding toreduce a bandwidth requirement of communication. For example, acompression algorithm, such as JPEG or JPEG2000, may be used to processimage data, and numerical data may be calculated by arithmetic coding.Control data may be processed by using Huffman coding.

The sensor signal processing module 404 may obtain effective movementsensing data P_(k) ^(jt) corresponding to each sensor from the signaltransmission processing device 403 and may calculate the position wherethe target object is most likely to be located in the target site area,and obtain the position information in the space motion informationbased on the movement sensing data P₁ ^(jt), . . . , P₁ ^(jt).

The position and orientation computing module 405 may overlay theangle^(t) data in the movement sensing data based on the positioninformation calculated by the sensor signal processing module 404, andobtain position information and rotating angle information of the targetobject in the space of the target site area by calculation.

The virtual object database 407 records three-dimensional models andtemplate data of various objects and virtual objects in the target sitearea. There are various shapes of real-world objects. For example, thehuman body may be tall, short, fat or slim. Therefore, it is necessaryto provide some template models with parameters, which may be used tosimply input parameters to obtain object models that are similar inshape and structure to real-world objects. For example, input height,weight and gender, a character model that meets the requirements may besimply constructed, which may be used as a representation of a virtualcharacter in the three-dimensional scene model corresponding to thetarget site area. In addition to the human body, simple objects, such ascylinders and cubes, may be included as a basis for object collisioninteraction. For the three-dimensional model of the virtual object, afiner object model which may represent a virtual object may beconfigured based on requirements to ensure visual effects during displayrendering.

The field of view computing module 406 may determine a user field ofview based on the position information and the field of view angle ofthe display device 402. Because a precision of the field of viewdetermines whether the user's visual experience is accurate, the fieldof view computing module 406 may perform refined calculation.Specifically, the field of view computing module 406 may determine,based on the sensed position of and rotating angle of the display device402 such as the AR glasses and with reference to an initial position andthe initial angle of the AR glasses, the change in position and thechange in angle, thereby obtaining a changed field of view. The shapeand size of the field of view may be a rectangle or polygon of a fixedsize value, however, the shape and size of the field of view are notlimited thereto. After obtaining the position information and the fieldof view of the display device 402, a target site area of the rectangleor polygon of the fixed size value may be selected from athree-dimensional spatial model representing the target site area, whichmay be an area that may be viewed in the user field of view. Inaddition, the obtained position and angle data of the display device 402may further be smoothed. Generally, smoothing should be performed incombination with logic of the change in field of view and the positionof the user's historical perspective. That is, the smoothing process canmake the change of the field of view relatively flat, which is suitablefor the user to view related content in the augmented reality scene. Forexample, when the user moves forward, the speed is uneven, but shouldconform to logic of the acceleration and deceleration of the human body,that is, the field of view change should be in a particular threshold.If this threshold is exceeded, it may be determined that the user isrunning or jumping. These changes in the motion state may be monitoredcontinuously to provide accurate representation of the sensed movementdata. For example, it is impossible to have 5 jumps in one second, andsmoothing is required.

The physical effect processing module 408 may be configured to connect areal object and virtual information in virtual space by using aclassical mechanics model, so that interactions between users objects inthe augmented reality scene conform to common scenes in daily life. Forexample, the main contents need to be processed may include rigid bodycollision, elastic deformation, free fall, explosion effect, and theothers. Because many objects in the real word do not actually collidewith objects in the virtual world, the physical effect processing module408 may set a three-dimensional rigid body corresponding to the realworld object disposed as a scene object during calculation, that is, themotion state thereof may not be changed.

The physical effect processing module 408 may need to input all fixedobjects, movable objects (including the user's body) and groundthree-dimensional models in the target site area as rigid body objects,which may be disposed as scene objects. In addition, three-dimensionalmodels of some virtual objects may be input to obtain athree-dimensional scene model about a target site area. Thethree-dimensional scene model may include all object models in thetarget site area, and some disposed object models of virtual objects,for example, a virtual character, a virtual tree and the like. Thespatial position of each object model in the three-dimensional scenemodel may be substantially the same as the spatial position of theobject corresponding to the object model in the target site area. Thephysical effect processing module 408 may dispose materials of variousobject models based on interaction requirements and calculateinteractions therebetween, by using classical mechanics formula. Theobject model may be updated in real time based on the position and thestate of objects in the target site area in the real world, so that eachobject model can effectively interact with each other. The interactionbetween the object model corresponding to the real object and the objectmodel corresponding to the virtual object may involve the interaction,such as collision that may exist between the two mentioned above.Accordingly, the physical effect processing module 408 is implemented byusing some physics engines.

Based on the calculation result of the physical effect processing module408 and the virtual model included in the three-dimensional scene model,the object model or some areas of the object model in the target area inthe three-dimensional scene model, are performed image rendering withthe field of range obtained by the field of view computing module 406with reference to virtual information required to be displayed. Thevirtual information required to be displayed may be converted into animage. For example, a real-time graphic data of the object model of thevirtual object may be converted into a virtual image that needs to beoverlay-displayed. The block of the real scene for the virtualinformation needs to be pre-calculated, and the rendering result may bemasked to exclude the blocked part of the virtual information, therebypreventing a mistake of generating overlay effect during display. Theimage rendering engine may use rendering functions in common gameengines.

The interactive control module 410 may be configured to process andcontrol interaction logic between the user and each object in the scene.For example, a user viewing a variety of virtual trees, displaying agraphical user interface (Graphical User Interface, GUI) for users tooperate. The interactive control module 410 may be disposed based onspecific requirements of actual AR application scene. For example, oneinterface may be reserved to implement subsequent functions.

The embodiments of the present disclosure may perform mobile monitoringof moving objects in a specific site area, and construct and obtain athree-dimensional scene model of the site area. Based on thethree-dimensional scene model with reference to the user positioncarrying the AR display device, virtual information of an augmentedreality scene already displayed on the user's display device may bequickly updated, and there is no need to use complex image analysis andrecognition algorithms to determine the display position of virtualinformation. Compared with the conventional processing way of directlyoverlaying virtual information in the captured image, in the embodimentsof the present disclosure, the process of blocking effect is added, sothat the blocking and the blocking effect may be better reflected in acase where the augmented reality scene is directly implemented based onthe position, so that the augmented reality scene is more realistic.According to an embodiment, the AR processing device and the AR displaydevice may be isolated. The user only needs to carry light AR glasseswith the display function, avoiding the problem that AR glasses arecumbersome because traditional AR glasses need to carry cameras. The ARprocessing device is generally a server that may display requirementsbased on the virtual information that needs to be processed by aplurality of users' augmented reality scenes simultaneously, and extendnew functions so that multiple users' requirements are in the augmentedreality scene, thereby saving costs to some extent.

FIG. 5 is a flowchart of a method for implementing an augmented realityscene according to an embodiment. The method may be performed by an ARprocessing device. Specifically, the method may be performed by acomputing device with an AR processing function, such as a server.Alternatively, the method may also be performed by an AR display deviceintegrated with a sensing data processing function and a displayfunction. The method may include the following steps.

In step S501, the method may include obtaining movement sensing data ofa target object acquired by a positioning apparatus, where thepositioning apparatus is configured to monitor the target object locatedin a target site area. The target site area may be a prearranged area,and may be arranged freely according to a user requirement for a scene.Specifically, step S501 may be triggered after a movement of the targetobject is detected. A gyroscope or an acceleration sensor disposed onthe target object may be used to determine whether the target object hasmoved.

The target site area may be a part of a large site area, or one of aplurality of arranged site areas. The site areas may be arranged,according to the requirement, to be a scene area of an indoor room, apark, a sports field or the like.

The target site area may include a fixed object and a mobile object. Forexample, in a site area of an indoor room scene, objects such as acloset and a pillar are fixed. A position of the fixed object is knownin the site area, and may not need to be monitored by a positioningapparatus. The positioning apparatus may sense movement of the mobileobject, so as to monitor a position of the mobile object. The mobileobject may be an object, such as a chair or the like, in the site areaof the indoor room scene. The target object may be any mobile object inthe target site area.

The target object may be a rigid object or a flexible object. Positionand angle of the entire target object may be sensed by an accelerationsensor, an angle sensor and a distance ranging sensor (distance sensorincludes distance ranging sensor using ultrasonic, laser and the like),such as a laser ranging array, so as to obtain the movement sensing dataof the entire target object.

The position information of the target object may be acquired by using adistance measuring sensor, and/or a positioning sensor. Specifically,position information of the target object in the target site area afterthe target object moves may be determined according to distance data ofthe target object obtained by at least two distance sensors, and the atleast two sensors may be configured to monitor the target object in thetarget site area. Alternatively, the position information of the targetobject in the target site area after the target object moves may beobtained by calculation according to motion data sensed by thepositioning sensor disposed on the target object.

In addition, when the target object is a flexible object, position of atarget object within a relatively low range of height from a ground or apart of the target object may be sensed by a ground auxiliarypositioning apparatus. The ground auxiliary positioning apparatus mayscan, using methods such as laser array scanning, ultrasonic arrayscanning or the like, within a specific range, for example, within 20 cmof the height from a ground, to obtain data related to the positioninformation of the target object or a partial structure of the targetobject in the target site area.

In step S502, the method may include determining, according to themovement sensing data, space motion information of the target object inthe target site area, and update, according to the space motioninformation, an object model of the target object in a three-dimensionalscene model corresponding to the target site area. The three-dimensionalscene model of the target site area may be a space model. Thethree-dimensional scene model may include object models of all objectswithin a specific spatial range of the target site area, and positionrelationships between objects corresponding to the object models.Specifically, a three-dimensional scene model of which an arrangement isthe same as that of the target site area may be created according to aspecific percentage of reduction and spatial position areas of actualobjects in the target site scene. In the three-dimensional scene model,the arrangement of the object models is the same as that of thecorresponding objects in the target site area. That is, the spatialposition area occupied by each object model in the three-dimensionalscene model is the same as or merely reduced to a specific scale of thatoccupied by the corresponding object in the target site area. Thethree-dimensional scene model of the target site area may bepre-configured. An object model of the mobile object exists in thethree-dimensional scene model. Therefore, the object model correspondingto the target object in the three-dimensional scene model may be updatedsubsequently based on a moving condition of the target object obtained.

The obtained movement sensing data, such as acceleration data and/orangular acceleration data of the three-dimensional space in the targetsite area, sensed by the positioning apparatus, may be processed toremove noise data to reduce error in position data and orientation data.The noise removing methods may include Fourier transform noise reductionand wavelet transform noise reduction algorithm. The position data andthe orientation data of the target object in the three-dimensional spacein which the target site area is located may be determined by using themovement sensing data. As such, a coordinate system of an actualthree-dimensional space in which the target site area is located may bemapped to a coordinate system corresponding to the three-dimensionalscene model corresponding to the target site scene. A position obtainedby the sensor and a rotation angle corresponding to the orientation datamay be mapped, according to the mapping relationship, to the coordinatesystem corresponding to the three-dimensional scene model. Based on aposition and an angle after mapping, and a basic parameter, such as ashape parameter of the object model corresponding to the target object,the object model corresponding to the target object in thethree-dimensional scene model may be remodeled, and an obtained objectmodel may be provided with a new position and a new angle in thecoordinate system in the three-dimensional scene model.

According to an embodiment, an update to the object model of the targetobject may include updating the position and/or a spatial orientation ofthe object model. If the space motion information includes the positioninformation of the target object after the target object moves, aposition of the object model of the target object in thethree-dimensional scene model may be updated according to the movedposition of the target object, and if the space motion informationincludes rotation angle information of the target object after thetarget object moves, the spatial orientation of the object model of thetarget object in the three-dimensional scene model may be updatedaccording to the rotated angle of the target object.

If the target object is a rigid object, after the target object moves,an original object model corresponding to the target object in thethree-dimensional scene model may be required to be moved or rotatedaccording to the position information and rotation angle information inthe movement sensing data.

If the target object is a flexible object, after the target objectmoves, position information and/or rotation angle information of aplurality of spots of the target object is obtained by calculation basedon the distance data that is obtained by different distance measuringsensors that are on different spots of the target object and based on analgorithm, such as triangulation position algorithm. If many spots onthe target object are moved and/or rotated, the object model of thetarget object in the three-dimensional scene model may be re-establishedbased on the moved and/or rotated spots. Moreover, if the target objectis a flexible object, after the target object moves, positions and/orangles of different components of the target object in thethree-dimensional space of the target site may be obtained bycalculation based on the position information and/or rotation angleinformation of the positioning sensor at different parts of the targetobject. The object model of the target object in the three-dimensionalscene model may be re-obtained by directly moving and rotating acorresponding component model in the object model of the target objectrespectively based on the position and the angle.

During the process of updating the object model of the target object,based on various movement sensing data described above, images may becombined to perform a more precise update on the object model. Thetarget site area may be further provided with at least two imagesensors, configured to monitor the target object from differentmonitoring angles to obtain image data of the target object. Step S502may further include updating, according to the space motion information,the object model of the target object in the three-dimensional scenemodel corresponding to the target site area to obtain an initiallyupdated object model, identifying an image object of the target objectfrom an image acquired by the at least two image sensors, and modifyingthe initially updated object model according to the identified imageobject to obtain an updated object model of the target object. Theinitially updated object model may be obtained by updating, based on theforegoing position information and/or rotation angle information in thespace motion information, the position and orientation of the objectmodel of the target object of the object model of the target object inthe three-dimensional scene model.

In step S503, the method may include determining a target objectposition of the updated object model in the target area. The target areamay be an area determined in the three-dimensional scene model accordingto position information and field of view information of the displaydevice in the target site area, and the determined target area may beconsidered as a field-of-view area. It may be considered that the objectmodel included in the target area (or a part of a model area of theobject model) corresponding to the object (or a part of the object) maybe seen by a user.

According to an embodiment, a determining manner of the target area mayinclude obtaining the position information and the field of viewinformation of the display device in the target site area, where thedisplay device is configured to display virtual information of anaugmented reality scene, determining a user field of view according tothe position information and the field of view information, anddetermining the target area in the established three-dimensional scenemodel according to the user field of view. For a head-mounted displaydevice, such as AR glasses, after the startup by a user, the positioninformation and the field of view information of the display device maybe sensed based on an angle sensor, such as the positioning sensor and agyroscope. A rectangular frame of a fixed size or a polygon of a fixedsize according to frame ranges of different VR glasses, such as theshape of the frame 301 of the AR glasses in FIG. 3 a, may bepre-configured to represent a field-of-view range. The field of viewinformation may represent angle information after rotation relative toinitial field-of-view orientation. After the position and afield-of-view angle are determined, a new field-of-view orientation maybe obtained by rotating the initial field-of-view orientation accordingto an angle indicated by the field of view information. The target areathat should appear in the field-of-view range of the user in thethree-dimensional scene model may be framed out based on the determinedposition and the new field-of-view orientation, and according to therectangular box or the polygon of the fixed size.

Moreover, the obtaining the position information and the field of viewinformation of the display device in the target site area may includereceiving the position information obtained after the positioningapparatus locates the display device, in which the positioning apparatusmay include a positioning sensor disposed on the display device, and/ora distance sensor that is disposed in the target site area, configuredto locate the display device, and receiving the field of viewinformation of the display device sensed by an angle sensor disposed onthe display device. The positioning apparatus may be an accelerationsensor, and may obtain, by sensing a tri-axial acceleration and time,and displacement of the display device relative to the initial position.The distance sensor may obtained the displacement of the display deviceby using triangle localization algorithm based on the displacementdistance of the display device sensed by at least two distance sensors.

The target area in the three-dimensional scene model may also include anobject model of another object other than the target object. After theupdate is completed, a position relationship between the updated objectmodel and the target area is determined. If the position relationshipindicates that any position area of the updated object model is not inthe target area, then the detection of the movement of a new targetobject may be continued. If the position relationship indicates that allor a part of areas of the updated object model is in the target area, anobject position of the object model of the target object in the targetarea may be further determined so as to perform step S504 describedherein below. The object position may be in the target area, and mayrefer to a position in which the entire object model of the targetobject or a part of the target object that is in the target area. Forexample, as shown in FIG. 3 b, a small desk is a target object, and apart of an object model of the small desk is shown in the frame of theAR glasses. The field-of-view range area of the frame corresponds to thetarget area in the three-dimensional scene model, and the objectposition is the position of a part of the object 302 located in theframe corresponding to the target area.

In step S504, the method may include displaying virtual information onthe display device according to the object position, so as to displaythe virtual information in the augmented reality scene. The obtainedobject position may be used to determine whether the virtual informationis influenced, for example, whether the target object in the objectposition blocks the virtual information or whether it is required todisplay the virtual information to the user in a position correspondingto the object position.

Specifically, if it is required to present the user with the virtualinformation overlaid on the target object, the object position may bemapped to a display area of the display device according to a mappingrelationship between the target area and a lens area of the displaydevice, then a new piece of virtual information may be generated, andthe generated virtual information is presented to the user by projectionon the display area, so that the user may see the augmented realityscene through the display device, such as AR glasses. Further, thecurrently displayed virtual information on the display device may beupdated to obtain new virtual information, and the updated virtualinformation may be displayed. The update may include a mask processingperformed to the currently displayed virtual information in a case thatthe target object moves onto the object position and blocks thecurrently displayed virtual information and/or a blocking removalperformed to the currently displayed virtual information in a case thatthe target object moves onto the object position so that the targetobject does not block the part of an area of the currently displayedvirtual information. If it is determined, according to the objectposition, that the movement of the target object does not have theforegoing influence on the displayed virtual information orto-be-displayed virtual information, the displaying virtual informationmay be refreshed, according to a normal generation and display refreshfrequency, to display the to-be-displayed virtual information orcontinue displaying the currently displayed virtual information.

The displaying of the virtual information may be performed in aplurality of manners. For example, a to-be-displayed image including thevirtual information may first be generated, and each pixel of theto-be-displayed image may be rendered to adjust a color value of eachpixel. The to-be-displayed image after rendering may be used as an imageoverlay layer. The image overlay layer may be projected to thetranslucent lens of the AR glasses by invoking the projector disposed onthe AR glasses, focusing on a retina by reflection of the translucentlens, to be captured by human eyes. When the image overlay layerincluding the virtual information is captured, the real target site areamay also be seen through the translucent lens.

Moreover, the color value of each pixel after the adjustment may bedetermined according to environment information of the target site area.Different environment information may perform different adjustments tothe color value of the pixels, so that the to-be-displayed imagematches, when finally presented to the user, with an environment of thetarget site area. The environment information may include information,such as a temperature and/or light intensity of the target site area.For example, in an environment with relatively high light intensity, animage obtained after adjusting the pixels presents an overall brightercolor.

After determining the object position of the object model and the targetobject in the target area, it may be required to determine whether theobject model, in the object position, and the target object block thecurrently displayed virtual information on the display device. If theobject model blocks the currently displayed virtual information, ablocked part in the virtual information may be required to be removed,and partial virtual information of the blocked part may not bedisplayed. For example, a part of the tree at a right side of FIG. 3b isactually blocked by the part of the object 302. Also, the object modelmay no longer block the virtual information when the target object ismoved, and in this case, the previously blocked part of the virtualobject is required to be updated and displayed.

Furthermore, when the foregoing steps are performed by a server havingan AR processing function, the method may also include sending a displaymessage to the display device, where the display message carries thevirtual information to be displayed on the display device and isconfigured to instruct the display device to display the carried virtualinformation, to overlay-display the generated virtual information in thetarget area covered by the user field of view and implement displayingthe augmented reality scene.

The embodiments of the present disclosure may monitor a moving object ina specific range of a site area, update an object model of the movingobject in a three-dimensional scene model corresponding to the site areain time after the object moves, generate, according to the updatedthree-dimensional scene model, virtual information to be displayed by adisplay device, and implement, in combination with a real site area seenthrough the display device, to display an augmented reality scene. Themethod does not require a complex image analysis recognition algorithmto determine a display position of the virtual information, and thecalculation may be performed quick and easy, thereby improvingefficiency of implementing an augmented reality scene.

FIG. 6 is a flowchart of a position information determining method basedon distance data according to an embodiment. The method corresponds tostep S501 in the foregoing embodiment, and is one of implementations forobtaining the position information. The method may include the followingsteps.

In step S601, the method may determine distance data about the targetobject, obtained by at least two proximity sensors. The at least twoproximity sensors may be configured to monitor the target object in thetarget site area. The proximity sensor may further include a distancemeasuring sensor using ultrasonic, laser or the like. Here, theproximity sensor may be dedicated to measuring a distance of the targetobject. For example, a receiver corresponding to the distance measuringsensor, using ultrasonic, laser or the like, may be disposed in aplurality of spots on the target object. The receiver may obtain a timeof receiving the ultrasonic and the laser, and the time may be fed backto the proximity sensor. Accordingly, the distance data is determined bythe proximity sensor.

In step S602, the method may include obtaining sensor positioninformation of the at least two proximity sensors. The positioninformation of the proximity sensors are pre-configured data, and thesensor position information may also be referred to as positioninformation in the target site area.

In step S603, the method may include performing least square fittingcalculation on the obtained distance data and sensor position data toobtain the position information of the target object in the target sitearea after the target object is moved.

For k proximity sensors, each piece of the valid distance data P^(jt) isobtained, and the most possible position in the target site area iscalculated according to the pieces of the distance data P₁ ^(jt), . . ., P_(k) ^(jt). A specific algorithm requires to input a coordinatePOS_(k) (such as a spatial position of a sensor such as a laserpositioning transmitter) of each positioning apparatus on the targetsite area, and is calculated by the least square fitting:

${{pos}_{j}^{*} = {\arg \; \min {\sum\limits_{pos}\left( {{pos}_{k}^{jt} - {{dist}\left( {{POS}_{k},{pos}} \right)}} \right)^{2}}}};$

An optimal position pos*_(j) of a monitored object in the target sitearea is obtained by estimation, where the dist function represents adistance function, and a three-dimensional Euclidean distance isgenerally used. POS_(k) is known, and pos is an auxiliary parameter inthe least square fitting calculation.

During a process of displaying the virtual information in the augmentedreality scene, the position of the target object may be preciselycalculated through the foregoing calculation manner. In particular,based on distance data of a plurality of proximity sensors, the positionof the target object can be obtained more quickly and accurately.

FIG. 7 is a flowchart of a virtual information updating method accordingto an embodiment. FIG. 7 relates to step S504 in the foregoingembodiment, and is one of virtual information updating methods. Themethod may include the following steps.

In step S700, the method may include determining target virtualinformation to be displayed on the display device. The determined targetvirtual information may be new virtual information required to beoverlay-displayed in the augmented reality scene so that the user mayview through the display device, such as AR glasses. One or more piecesof information required to be displayed in a superimposed manner and/orimages may be selected, as the target virtual information, frompre-configured content, such as pieces of information and images. Inaddition, the determined target virtual information may also correspondto the currently displayed virtual information on the display device.For example, the currently displayed virtual information may be about atreetop part of a virtual tree image. The virtual information of thecurrently shown treetop part may correspond to the virtual tree image,and the currently determined virtual tree image may also be used as thetarget virtual information, so as to subsequently perform a processingstep of updating the currently displayed virtual information because ofthe movement of the target object.

In step S701, the method may include detecting a blocking relationshipbetween the object model of the target object and the target virtualinformation according to the object position. The blocking relationshipbetween the object model and the target virtual information may bedetermined according to an object position and a shape of the objectmodel, a position of a target virtual information to be displayed and ashape of the target virtual information, and a position and anorientation of the display device. Specifically, the object position maycorrespond to a spatial position area occupied by the target object inthe target site area, and is not simply a position spot.

In step S702, the method may include determining, in a case that theblocking relationship indicates that the object model of the targetobject blocks the virtual information, a first area that is blocked inthe target virtual information, and obtain first virtual information,where the first virtual information does not include contentcorresponding to the first area of the target virtual information. Asshown in FIG. 3 b, the virtual tree, as the virtual information 108, isblocked by a small desk, for example, a part of the object 302. The partof the object 302 is considered as a first area, and when the targetvirtual information is processed, an image content of this part isrequired to be removed. The first virtual information is virtualinformation that is finally generated and to be displayed on the displaydevice.

In step S703, the method may include determining, in a case that theblocking relationship indicates that the object model of the targetobject does not block a second area of the target virtual information,obtain second virtual information, where the second virtual informationincludes content corresponding to the second area of the target virtualinformation. For example, if the small desk is moved away by anotheruser, according to a new object position of the object modelcorresponding to the small desk, it is determined that the object modelof the small desk no longer blocks the virtual tree (the virtualinformation 108). The content of the blocked second area, for example,the part of the object 302, is determined so as to obtain the secondvirtual information of the content corresponding to the second area. Assuch, the second virtual information is the virtual information that isfinally generated and to be displayed on the display device.

Compared with a conventional processing manner of directly superimposingthe virtual information on a captured image, the embodiments of thepresent disclosure adds the blocking processing, so that the embodimentscan better show effects of blocking and blocking removal in a case thatthe augmented reality scene is directly implemented based on theposition, allowing the augmented reality scene to be more realistic.

FIG. 8 is a flowchart of another method for implementing an augmentedreality scene according to an embodiment, and the method may beperformed by an AR display device. The AR display device may update thedisplay of virtual information in the augmented reality scene throughinteraction with the AR processing device, and overlay-display thevirtual information in a user field of view to display the augmentedreality scene. The method may include the following steps.

In step S801, the method may include obtaining the position informationand the field of view information of the display device in the targetsite area. The display device may be configured to display the virtualinformation of the augmented reality scene. The position information andthe field of view information of the display device may be sensed basedon an angle sensor, such as a positioning sensor or a gyroscope.

In step S802, the method may include generating orientation indicationinformation including the position information and the field of viewinformation, and send the orientation indication information to theaugmented reality processing device. The orientation indicationinformation may include virtual information that is generated by theaugmented reality processing device and that is to be displayed in theuser field of view determined by the position information and the fieldof view information. The manner in which the augmented realityprocessing device obtains the virtual information or updates virtualinformation according to the position information and the field of viewinformation, may be referenced to the description of relevant contentsin the foregoing embodiments.

In step S803, the method may include receiving a display message that issent by the augmented reality processing device and that carries thevirtual information, and display the virtual information according tothe display message, so as to display the virtual information in theuser field of view in a superimposed manner to complete displaying ofthe augmented reality scene. The augmented reality display device mayonly display the virtual information or the updated virtual information.A real object in the target site area may be seen through the augmentedreality display device, and the virtual information may be displayed ina superimposed manner.

The step S803 may further include receiving an image sequence and animage timestamp sequence sent by the augmented reality processingdevice, obtaining a time value of a current moment, and deleting, in acase that a target timestamp exists in the received image timestampsequence, an image frame corresponding to the target timestamp from theimage sequence. The target timestamp may include a timestamp, adifference between the timestamp and the time value of the currentmoment greater than twice an image display refreshing period of thedisplay device in the image timestamp sequence.

To-be-displayed virtual information received by the display device at amoment t includes an image sequence I, where I=(img_(t1), img_(t1), . .. , img_(tn)), and the timestamp T_(s) corresponding to each piece ofthe image data in the image sequence, where T_(s)=(t_(s) ¹, t_(s) ², . .. , t_(s) ^(k)), and the image display refreshing period of the displaydevice is t_(s). Here, if the current moment t−t_(s) ^(i)>2 t_(s)represents that a next image frame of the i^(th) frame has beencurrently received, the i^(th) frame img_(ti) may be discarded directly.If a case that t−t_(s) ^(i)>2 t_(s) exists, the discarded i^(th) imagemay be predicted through a difference prediction manner. A method inwhich img_(ti)=img_(ti−1)+Δimg may be used for pre-calculation to reduceeffects of freezing and lagging in the displayed image on the screencaused by transmission. Δimg is obtained through calculation accordingto a pixel value of relevant pixels in image data of a previous frameand a next frame of the i^(th) frame. A specific calculation manner mayuse a piecemeal motion interpolation method, and a specific algorithmmay be an H.265 compression algorithm standard. Furthermore, smoothprocessing of a frame rate may be performed to data of differentmoments, and when necessary, difference prediction may be performed onthe content of the image.

According to an embodiment, the AR processing device may be separatedfrom the AR display device, and the user only needs to carry a light ARdisplay device having a display function, such as AR glasses, therebypreventing a problem that AR glasses are relatively cumbersome becausethe conventional AR glasses are required to carry a camera. In addition,the AR display device may suitably discard a part of data from thereceived to-be-displayed virtual information to implement smoothprocessing quickly, and further prevent lagging and freezing.

FIG. 9 is a schematic structural diagram of an apparatus forimplementing an augmented reality scene according to an embodiment. Theapparatus may be disposed in an AR processing device, for example, in aserver having an augmented reality function. The apparatus may includean obtaining module 901 configured to obtain movement sensing data of atarget object acquired by a positioning apparatus, the positioningapparatus being configured to monitor the target object located in atarget site area; a model update module 902 configured to determine,according to the movement sensing data, space motion information of thetarget object in the target site area, and update, according to thespace motion information, an object model of the target object in athree-dimensional scene model corresponding to the target site area; adetermining module 903 configured to determine an object position of theupdated object model in a target area, the target area being an areadetermined in the three-dimensional scene model according to positioninformation and field of view information of the display device in thetarget site area; and a display module 904 configured to display virtualinformation on the display device according to the object position, todisplay the virtual information in the augmented reality scene.

The positioning device may include various sensors described above, andmay include a proximity sensor, a positioning sensor, and a groundassisted positioning device and the like. Specifically, the device mayinclude an angle speed sensor, an angle sensor, an infrared rangingsensor, and a laser ranging sensor. These sensors may be disposed in thetarget site area, or may be disposed on each target object of the targetsite area, which are configured to perform movement sensing on eachtarget object to obtain corresponding movement sensing data.

The three-dimensional scene model of the target site area is a spacemodel. The three-dimensional scene model may include object models ofall objects within a specific spatial range of the target site area, andposition relationships between objects corresponding to the objectmodels.

The model update module 902 may be configured to update the object modelof the target object by updating the position and/or a spatialorientation of the object model. If the target object is a rigid object,after the target object moves, an original object model corresponding tothe target object in the three-dimensional scene model may be requiredto be moved or rotated according to the position information androtation angle information in the movement sensing data. If the targetobject is a flexible object, the three-dimensional scene model may berequired to sense a plurality of position points to obtain relevantsensing data of the entire target object and update the entire objectmodel.

The display module 904 may be configured to present the user the virtualinformation superimposed on the target object. As such, the objectposition may be mapped to a display area of the display device accordingto a mapping relationship between the target area and a lens area of thedisplay device, new virtual information may be generated, and thegenerated virtual information may be presented to the user by projectionon the display area, so that the user may view the augmented realityscene through the display device. The display module 904 may be furtherconfigured to update the currently displayed virtual information on thedisplay device to obtain virtual information, and display the updatedvirtual information. The update may include a mask processing performedto the currently displayed virtual information in a case that the targetobject moves to the object position and blocks the currently displayedvirtual information, or a blocking removal performed to the currentlydisplayed virtual information in a case that the target object moves tothe object position and does not block the part of area of the currentlydisplayed virtual information. If it is determined, according to theobject position, that the movement of the target object does not havethe foregoing influence on the displayed virtual information orto-be-displayed virtual information, the display module 904 may befurther configured to refresh, according to a normal generation anddisplay refresh frequency, displaying of the to-be-displayed virtualinformation or previously displayed virtual information.

After determining, by the determining module 903, the position of theobject model of the target object in the target area, the display module904 may determine whether the object model of the target object blocksthe currently displayed virtual information of the display device. Ifthe object model blocks the currently displayed virtual information, ablocked part in the virtual information is required to be removed, andvirtual information of the blocked part is not displayed. For example, apart of the tree on a right side of FIG. 3b is blocked by the part ofthe object 302. When the object model of the target object is moved andno longer blocks the virtual information, the blocked part of thevirtual object is required to be updated and displayed.

In an embodiment, the space motion information may include positioninformation and/or rotation angle information of the target object aftermoving, and the model update module 902 may be configured to update, ina case that the space motion information includes the positioninformation of the target object after the target object moves, aposition of the object model of the target object in thethree-dimensional scene model according to the position information andupdate, in a case that the space motion information includes therotation angle information of the target object after moving, spatialorientation of the object model of the target object in thethree-dimensional scene model according to the rotation angleinformation.

A first position obtaining module 907 may be configured to obtaindistance data of the target object by at least two proximity sensors,and determine the position information of the target object in thetarget site area after the target object moves, the at least twoproximity sensors being configured to monitor the target object in thetarget site area. A second position obtaining module 908 may beconfigured to calculate, according to motion data sensed by apositioning sensor disposed on the target object, the positioninformation of the target object in the target site area after thetarget object moves. A third position obtaining module 909 may beconfigured to calculate, according to data sensed by a ground auxiliarypositioning apparatus disposed in the target site area, positioninformation of the target object or a partial structure of the targetobject in the target site area, a height of the target object or thepartial structure of the target object from a ground falling within apreset height threshold range. According to an embodiment, any one of ora plurality of modules including the first position obtaining module907, the second position obtaining module 908 and the third positionobtaining module 909 may be implemented according to a practicalrequirement and an arrangement of positioning apparatus.

The first position obtaining module 907 may be further configured todetermine distance data about the target object obtained by at least twoproximity sensors, obtain sensor position information of the at leasttwo proximity sensors, and perform least square fitting calculation onthe obtained distance data and sensor position information to obtain theposition information of the target object in the target site area afterthe target object moves.

In an embodiment, the target site area may be further provided with atleast two image sensors configured to monitor the target object fromdifferent monitoring angles to obtain image data of the target object.The model update module 902 may be further configured to update,according to the space motion information, the object model of thetarget object in the three-dimensional scene model corresponding to thetarget site area to obtain an initially updated object model. An imageobject of the target object may be identified from an image acquired bythe at least two image sensors, and the initially updated object modelmay be corrected according to the identified image object to obtain anupdated object model of the target object.

A determining module 905 may be configured to obtain the positioninformation and the field of view information of the display device inthe target site area, the display device being configured to display thevirtual information of the augmented reality scene, Here, a user fieldof view may be determined according to the position information and thefield of view information, and the target area may be determined in theestablished three-dimensional scene model according to the user field ofview.

Further, the determining module 905 may be configured to receive theposition information obtained after the positioning apparatus locatesthe display device, where the positioning apparatus includes apositioning sensor disposed on the display device, and/or a proximitysensor that is disposed in the target site area and configured to locatethe display device and receive the field of view information of thedisplay device sensed by an angle sensor disposed on the display device.

According to an embodiment, the display module 904 may be furtherconfigured to determine target virtual information to be displayed onthe display device, detect a blocking relationship between the objectmodel of the target object and the target virtual information accordingto the object position, determine, in a case that the blockingrelationship indicates that the object model of the target object blocksthe virtual information, a first area that is blocked in the targetvirtual information, and obtain first virtual information. Here, thefirst virtual information does not include a content corresponding tothe first area of the target virtual information. The display module 904may also determine, in a case that the blocking relationship indicatesthat the object model of the target object does not block a second areaof the target virtual information, the second area of the target virtualinformation, and obtain second virtual information, where the secondvirtual information includes a content corresponding to the second areaof the target virtual information.

A transmission module 906 may be configured to send a display message tothe display device, where the display message carries the virtualinformation to be displayed on the display device, and may be configuredto instruct the display device to display the carried virtualinformation to overlay-display the generated virtual information in thetarget area covered by the user field of view to complete the displayingof the augmented reality scene. The virtual information obtained by thetransmission module 906 may be sent, to an AR display device to instructthe AR display device to display the virtual information in acorresponding position.

The embodiments of the present disclosure may monitor a moving object ina specific range of a site area, update an object model of the movingobject in a three-dimensional scene model corresponding to the site areain time after the object moves, generate, according to the updatedthree-dimensional scene model, virtual information to be displayed by adisplay device, and implement, in combination with a real site area seenthrough the display device displaying an augmented reality scene. Themethod does not require a complex image analysis recognition algorithmto determine a display position of the virtual information, andcalculation is quick and easy, thereby improving efficiency ofimplementing an augmented reality scene.

FIG. 10 is a schematic structural diagram of another apparatus forimplementing an augmented reality scene according to an embodiment. Theapparatus may be disposed in an AR display device, for example, in ARglasses having a function of displaying relevant virtual information ofthe augmented reality scene. The apparatus may include an obtainingmodule 1001 configured to obtain the position information and the fieldof view information of the display device in the target site area, thedisplay device being configured to display the virtual information ofthe augmented reality scene; a generation module 1002 configured togenerate orientation indication information carrying the positioninformation and the field of view information, and send the orientationindication information to the augmented reality processing device, theorientation indication information being used to indicate the virtualinformation that is generated by the augmented reality processing deviceand that is to be displayed in the user field of view determined by theposition information and the field of view information; and a processingmodule 1003 configured to receive a display message that is sent by theaugmented reality processing device and that carries the virtualinformation, and display the virtual information according to thedisplay message to overlay-display the virtual information in the userfield of view to complete displaying of the augmented reality scene.

The processing module 1003 may be further configured to receive an imagesequence and an image timestamp sequence sent by the augmented realityprocessing device, obtain a time value of a current moment, and delete,in a case that a target timestamp exists in the received image timestampsequence, an image frame corresponding to the target timestamp from theimage sequence. The target timestamp may include a timestamp, adifference between which and the time value of the current moment isgreater than twice an image display refreshing period of the displaydevice in the image timestamp sequence.

Here, the AR processing device may be separated from the AR displaydevice, and the user only needs to carry a light AR display device witha display function, such as AR glasses and the like, thereby preventinga problem that AR glasses are relatively cumbersome because theconventional AR glasses are required to carry a camera. In addition, theAR display device may suitably discard a part of data from the receivedto-be-displayed virtual information, implement smooth processingquickly, and further prevent lagging and freezing.

FIG. 11 is a schematic structural diagram of an augmented realityprocessing device according to an embodiment. The augmented realityprocessing device of this embodiment may be a computing device, such asa server, and may include a structure, such as a power supply apparatus,a heat dissipation apparatus. Furthermore, the augmented realityprocessing device may include a storage apparatus 1101, a processor 1102and a communication interface 1103.

The communication interface 1103 may be a wired interface or a wirelessinterface. On one hand, the communication interface may receive movementsensing data of the target object in the target site area and that isuploaded by various positioning apparatuses. On the other hand, thecommunication interface may send the virtual information processed bythe processor 1102 to the AR display device in the target site area.

The storage apparatus 1101 may include a volatile memory and anon-volatile memory. The storage apparatus 1101 may further include acombination of the foregoing types of memories.

The processor 1102 may be a central processing unit (CPU). The processor1102 may further include a hardware chip.

The storage apparatus 1101 may store a computer application instruction.The processor 1102 may invoke the computer application instructionstored in the storage apparatus 1101 to perform the relevant method ofthe embodiments herein.

According to an embodiment, the processor 1102 may invoke the storedcomputer application instruction to perform the following operations:obtaining movement sensing data of a target object acquired by apositioning apparatus, the positioning apparatus being configured tomonitor the target object located in a target site area; determining,according to the movement sensing data, space motion information of thetarget object in the target site area, and updating, according to thespace motion information, an object model of the target object in athree-dimensional scene model corresponding to the target site area;determining an object position of the updated object model in a targetarea, the target area being an area determined in the three-dimensionalscene model according to position information and field of viewinformation of the display device in the target site area; anddisplaying virtual information on the display device according to theobject position, to display the virtual information in the augmentedreality scene.

According to an embodiment, the space motion information may includeposition information and/or rotation angle information of the targetobject after moving, and the processor 1102, when configured to update,according to the space motion information, an object model of the targetobject in a three-dimensional scene model corresponding to the targetsite area, may be configured to perform the following operations:updating, in a case that the space motion information includes theposition information of the target object after moving, a position ofthe object model of the target object in the three-dimensional scenemodel according to the position information; and updating, in a casethat the space motion information comprises the rotation angleinformation of the target object after moving, spatial orientation ofthe object model of the target object in the three-dimensional scenemodel according to the rotation angle information.

According to an embodiment, the acquisition of position information by aprocessor 1102 may include performing any one of or a combination of thefollowing operations: obtaining distance data of the target objectaccording to at least two proximity sensors, and determining theposition information of the target object in the target site area afterthe target object moves, the at least two proximity sensors beingconfigured to monitor the target object in the target site area;calculating, according to motion data sensed by a positioning sensordisposed on the target object, the position information of the targetobject in the target site area after the target object moves; andcalculating, according to data sensed by a ground auxiliary positioningapparatus disposed in the target site area, position information of thetarget object or a partial structure of the target object in the targetsite area, a height of the target object or the partial structure of thetarget object from a ground falling within a preset height thresholdrange.

According to an embodiment, the processor 1102, when performing theobtaining distance data of the target object according to at least twoproximity sensors, and determining the position information of thetarget object in the target site area after the target object moves, mayperform the following operations: determining the distance data of thetarget object obtained by the at least two proximity sensors; obtainingsensor position information of the at least two proximity sensors; andperforming least square fitting calculation on the obtained distancedata and sensor position information to obtain the position informationof the target object in the target site area after the target objectmoves.

According to an embodiment, the target site area may be further providedwith at least two image sensors configured to monitor the target objectfrom different monitoring angles to obtain image data of the targetobject. The processor 1102 may be configured to update, according to thespace motion information, an object model of the target object in athree-dimensional scene model corresponding to the target site area, andmay further perform the following operations: updating, according to thespace motion information, the object model of the target object in thethree-dimensional scene model corresponding to the target site area toobtain an initially updated object model; and identifying an imageobject of the target object from an image acquired by the at least twoimage sensors, and correcting the initially updated object modelaccording to the identified image object to obtain an updated objectmodel of the target object.

According to an embodiment, before performing the obtaining movementsensing data of a target object acquired by a positioning apparatus, theprocessor 1102 may further perform the following operations: obtainingthe position information and the field of view information of thedisplay device in the target site area, the display device beingconfigured to display the virtual information of the augmented realityscene; and determining a user field of view according to the positioninformation and the field of view information, and determining thetarget area in the established three-dimensional scene model accordingto the user field of view.

According to an embodiment, when performing the obtaining the positioninformation and the field of view information of the display device inthe target site area, the processor 1102 may perform the followingoperations: receiving the position information obtained after thepositioning apparatus locates the display device, wherein thepositioning apparatus comprises a positioning sensor disposed on thedisplay device, and/or a proximity sensor that is disposed in the targetsite area and configured to locate the display device; and receiving thefield of view information of the display device sensed by an anglesensor disposed on the display device.

According to an embodiment, when performing the displaying virtualinformation on the display device according to the object position, theprocessor 1102 may perform the following operations: determining targetvirtual information to be displayed on the display device; detecting ablocking relationship between the object model of the target object andthe target virtual information according to the object position;determining, in a case that the blocking relationship indicates that theobject model of the target object blocks the virtual information, afirst area that is blocked in the target virtual information, and obtainfirst virtual information, where the first virtual information does notinclude a content corresponding to the first area of the target virtualinformation; and determining, in a case that the blocking relationshipindicates that the object model of the target object does not block asecond area of the target virtual information, the second area of thetarget virtual information, and obtain second virtual information, wherethe second virtual information includes a content corresponding to thesecond area of the target virtual information.

According to an embodiment, the processor 1102 may further perform thefollowing operation: sending a display message to the display device,wherein the display message carries the virtual information to bedisplayed on the display device, and may be configured to instruct thedisplay device to display the carried virtual information tooverlay-display the virtual information in the target area covered bythe user field of view to complete the displaying of the augmentedreality scene.

The embodiments of the present disclosure may monitor a moving object ina specific range of a site area, update an object model of the movingobject in a three-dimensional scene model corresponding to the site areain time after the object moves, generate, according to the updatedthree-dimensional scene model, virtual information to be displayed by adisplay device, and implement, in combination with a real site area seenthrough the display device, displaying an augmented reality scene. Themethod does not require a complex image analysis recognition algorithmto determine a display position of the virtual information, andcalculation is quick and easy, thereby improving efficiency ofimplementing an augmented reality scene.

FIG. 12 is a schematic structural diagram of an augmented realitydisplay device according to an embodiment. The augmented reality displaydevice may be a computing device that may display virtual information ofan augmented reality scene. This device may include a structure, such asa power supply apparatus, a heat dissipation apparatus and the like. Inan embodiment, the display device may be AR glasses. The shape of thisdevice may be as shown in FIG. 2, but the shape of the device is notlimited thereto. The augmented reality display device may include astorage apparatus 1201, a processor 1202, a display screen 1203, anetwork interface 1204, a user interface 1205, and a sensor 1206.

The display screen 1203 may be a transparent display screen. Through thetransparent display screen 1203, a real object in the real word may beseen and the virtual information may be displayed in the augmentedreality scene. The display screen 1203 may be a display screen 1203 madeof an organic light-emitting diode (OLED) material.

The network interface 1204 may be configured to receive the virtualinformation transmitted by an augmented reality processing device. Themovement sensing data of the augmented reality processing device, forexample, the position information and the field of view information maybe transmitted to the display device through the network interface 1204.The position information and the field of view information may becalculated and obtained based on the data collected by the sensor 1206,such as an acceleration sensor and an angle sensor, for example, agyroscope, disposed in the augmented reality display device.

The user interface 1205 may include a physical button, touch pad or somestructures which may sense user's gesture, and may be configured toreceive some user's control instructions.

The storage apparatus 1201 may include a volatile memory and anon-volatile memory. The storage apparatus 1201 may further include acombination of the foregoing types of memories.

The processor 1202 may be a central processing unit (CPU). The processor1202 may further include a hardware chip.

The storage apparatus 1201 may store a computer application instruction.The processor 1202 may invoke the computer application instructionstored in the storage apparatus 1201 to perform the relevant method ofthe embodiments herein.

The processor 1202 may invoke the stored computer applicationinstruction to perform the following operations: obtaining the positioninformation and the field of view information of the display device inthe target site area, the display device being configured to display thevirtual information of the augmented reality scene; generatingorientation indication information carrying the position information andthe field of view information, and sending the orientation indicationinformation to the augmented reality processing device, the orientationindication information being used to indicate the virtual informationthat is generated by the augmented reality processing device and that isto be displayed in the user field of view determined by the positioninformation and the field of view information; and receiving a displaymessage that is sent by the augmented reality processing device and thatcarries the virtual information, and displaying the virtual informationaccording to the display message to overlay-display the virtualinformation in the user field of view to complete the displaying of theaugmented reality scene.

When performing the receiving a display message that is sent by theaugmented reality processing device and that carries the virtualinformation, the processor 1202 may be configured to perform thefollowing operations: receiving an image sequence and an image timestampsequence sent by the augmented reality processing device; obtaining atime value of a current moment; and deleting, in a case that a targettimestamp exists in the received image timestamp sequence, an imageframe corresponding to the target timestamp from the image sequence.Here, the target timestamp may include a timestamp, a difference betweenwhich and the time value of the current moment is greater than twice animage display refreshing period of the display device in the imagetimestamp sequence.

Here, the AR processing device may be separated from the AR displaydevice, and the user only needs to carry a light AR display device witha display function, such as AR glasses and the like, thereby preventinga problem that AR glasses are relatively cumbersome because theconventional AR glasses are required to carry a camera. In addition, theAR display device may suitably discard a part of data from the receivedto-be-displayed virtual information, implement smooth processingquickly, and further prevent lagging and freezing.

The embodiments of the present disclosure further provide a system forimplementing an augmented reality scene, as shown in FIG. 13. The systemmay include an augmented reality processing device 1301 and an augmentedreality display device 1302. The augmented reality processing device1301 may be a server, and the augmented reality display device 1302 maybe a display device which may display the virtual information of theaugmented reality scene. The display device may be a pair of AR glasses,and the shape of the device is shown in FIG. 2.

The augmented reality processing device 1301 may be configured to obtainmovement sensing data of a target object acquired by a positioningapparatus, and the positioning device may be configured to monitor thetarget object located in a target site area; determine, according to themovement sensing data, space motion information of the target object inthe target site area, and update, according to the space motioninformation, an object model of the target object in a three-dimensionalscene model corresponding to the target site area; determine an objectposition of the updated object model in a target area, the target areabeing an area determined in the three-dimensional scene model accordingto position information and field of view information of the displaydevice in the target site area; and displaying virtual information onthe display device according to the object position.

The augmented reality display device 1302 may be configured to obtainthe position information and the field of view information of thedisplay device in the target site area. The display device may beconfigured to display the virtual information of the augmented realityscene, generate orientation indication information carrying the positioninformation and the field of view information, and send the orientationindication information to the augmented reality processing device, theorientation indication information being used to indicate the virtualinformation that is generated by the augmented reality processing deviceand that is to be displayed in the user field of view determined by theposition information and the field of view information; and receive adisplay message that is sent by the augmented reality processing deviceand that carries the virtual information, and display the virtualinformation according to the display message, to overlay-display thevirtual information in the user field of view to complete displaying ofthe augmented reality scene.

According to the embodiments of the disclosure, the AR processing devicemay be separated from the AR display device, and the user only needs tocarry a light AR display device with a display function, such as ARglasses and the like, thereby preventing a problem that AR glasses arerelatively cumbersome because the conventional AR glasses are requiredto carry a camera. In addition, the AR display device may suitablydiscard a part of data from the received to-be-displayed virtualinformation, implement smooth processing quickly, and further preventlagging and freezing.

A person of ordinary skill in this art may understand that, all or someprocedures in the methods in the foregoing embodiments may beimplemented by a program instructing related hardware. The program maybe stored in a computer readable storage medium. When being executed,the program may include the procedures according to the embodiments ofthe foregoing methods. The storage medium may be a magnetic disk, anoptical disc, a read-only memory (ROM), a random access memory (RAM), orthe like.

The foregoing descriptions are merely some embodiments of thedisclosure, and are not intended to limit the scope of the disclosure. Aperson of ordinary skill in the art may understand all or some processesof the foregoing embodiments, and equivalent modifications madeaccording to the embodiments of the disclosure shall still fall withinthe scope of the disclosure.

What is claimed is:
 1. A method for implementing an augmented realityscene, executed by a computing device, the method comprising: obtainingmovement sensing data of a target object acquired by a positioningapparatus, the positioning apparatus being configured to monitor thetarget object in a target site area; determining, based on the movementsensing data, space motion information of the target object in thetarget site area; updating, based on the space motion information, anobject model of the target object in a three-dimensional scene modelcorresponding to the target site area to obtain a first updated objectmodel of the target object; and determining an object position of thefirst updated object model in a target area to display, on a displaydevice, virtual information according to the object position, the targetarea being an area determined in the three-dimensional scene model basedon position information and field of view information of the displaydevice in the target site area.
 2. The method according to claim 1,wherein the space motion information comprises at least one of positioninformation and rotation angle information of the target object, andwherein the method further comprises: based on the space motioninformation including the position information of the target object inthe three-dimensional scene model, updating a position of the objectmodel of the target object in the three-dimensional scene modelaccording to the position information; and based on the spaceinformation including the rotation angle information of the targetobject, updating spatial orientation of the object model of the targetobject in the three-dimensional scene model according to the rotationangle information.
 3. The method according to claim 2, whereindetermining the position information comprises: obtaining distance dataof the target object according to at least two proximity sensorsconfigured to monitor the target object, and determining a position ofthe target object in the target site area after the target object moves;calculating, according to motion data sensed by a positioning sensordisposed on the target object, the position of the target object in thetarget site area after the target object moves; and calculating,according to data sensed by a ground auxiliary positioning apparatusdisposed in the target site area, at least one of the position of thetarget object, a position of a partial structure of the target object inthe target site area, a height of the target object or a height of thepartial structure of the target object falling within a preset heightthreshold range.
 4. The method according to claim 3, wherein theobtaining distance data of the target object and determining theposition information of the target object further comprise: obtainingsensor position information of the at least two proximity sensors; andperforming least square fitting calculation on the obtained distancedata and sensor position information to obtain the position informationof the target object in the target site area.
 5. The method according toclaim 1, wherein the target site area comprises at least two imagesensors configured to monitor the target object from different angles toobtain image data of the target object; and wherein the updating theobject model of the target object in the three-dimensional scene modelfurther comprises: identifying an image object of the target object fromimages acquired by the at least two image sensors, and correcting thefirst updated object model according to the identified image object toobtain a second updated object model of the target object.
 6. The methodaccording to claim 1, wherein the obtaining movement sensing data of thetarget object further comprises: obtaining the position information andthe field of view information of the display device in the target sitearea, the display device being configured to display the virtualinformation of the augmented reality scene; and determining a user fieldof view according to the position information and the field of viewinformation, and determining the target area in the establishedthree-dimensional scene model according to the user field of view. 7.The method according to claim 6, wherein the obtaining the positioninformation and the field of view information of the display device inthe target site area comprises: receiving the position information afterthe positioning apparatus locates the display device, the positioningapparatus comprising at least one of a positioning sensor disposed onthe display device and a proximity sensor disposed in the target sitearea; and receiving the field of view information of the display devicesensed by an angle sensor disposed on the display device.
 8. The methodaccording to claim 1, wherein the determining the object position of thefirst updated object model in the target area further comprises:determining target virtual object to be displayed on the display device;detecting a blocking relationship between the object model of the targetobject and the target virtual object; based on the block relationshipindicating that the object model of the target object blocks the targetvirtual object, determining a first area that is blocked and obtainingfirst virtual information, wherein the first virtual information doesnot overlap in the first area; and based on the block relationshipindicating that the object model of the target object does not block thetarget virtual object, determining a second area and obtaining secondvirtual information corresponding to the second area.
 9. The methodaccording to claim 1, further comprising: sending a display message tothe display device, wherein the display message carries the virtualinformation to be displayed on the display device and instructs thedisplay device to display the carried virtual information to overlay thegenerated virtual information in the target area covered by the userfield of view.
 10. A method for implementing an augmented reality scene,executed by a display device, the method comprising: obtaining positioninformation and field of view information of the display device in atarget site area, the display device being configured to display virtualinformation of the augmented reality scene; generating orientationindication information based on the position information and the fieldof view information, and transmitting the orientation indicationinformation to an augmented reality processing device, the orientationindication information instructing the augmented reality processingdevice to generate the virtual information to be displayed in a userfield of view determined by the position information and the field ofview information; and receiving a display message carrying the virtualinformation, from the augmented reality processing device, anddisplaying the virtual information based on the display message, thedisplay message instructing the display device to overlay-display thevirtual information in the user field of view.
 11. The method accordingto claim 10, wherein the receiving the display message carrying thevirtual information further comprises: receiving an image sequence andan image timestamp sequence from the augmented reality processingdevice; obtaining a time value at a current moment; and based ondetecting a target timestamp in the received image timestamp sequence,deleting an image frame corresponding to the target timestamp from theimage sequence, wherein the target timestamp is a difference between atimestamp and the time value of the current moment, the difference beinggreater than twice an image display refresh period of the displaydevice, in the image timestamp sequence.
 12. A non-transitory computerstorage medium, storing executable instructions, the executableinstructions capable of causing a computer to perform the method forimplementing the augmented reality scene according to claim
 1. 13. Anaugmented reality display device, comprising: a storage apparatus and aprocessor, the storage apparatus storing a computer program, and theprocessor being configured to invoke the computer program stored in thestorage apparatus, to perform the method according to claim
 10. 14. Anon-transitory computer storage medium, storing executable instructions,the executable instructions capable of causing a computer to perform themethod for implementing the augmented reality scene according to claim10.
 15. An augmented reality display device, comprising: at least onememory configured to store computer program code; and at least oneprocessor configured to access the computer program code and operate asinstructed by the computer program code, the computer program codecomprising: obtaining code configured to cause the at least oneprocessor to obtain movement sensing data of a target object acquired bya positioning apparatus, the positioning apparatus being configured tomonitor the target object in a target site area; model update codeconfigured to cause the at least one processor to determine, based onthe movement sensing data, space motion information of the target objectin the target site area, and update, based on the space motioninformation, an object model of the target object in a three-dimensionalscene model corresponding to the target site area to obtain a firstupdated object model of the target object; determining code configuredto cause the at least one processor to determine an object position ofthe first updated object model in a target area, the target area beingan area determined in the three-dimensional scene model according toposition information and field of view information of the display devicein the target site area; and display code configured to cause the atleast one processor to display virtual information on the augmentedreality display device according to the object position.
 16. Theaugmented reality display device according to claim 15, wherein thespatial motion information comprises at least one of the positioninformation and rotation angle information of the target object; andwherein the updating code is further configured to cause the at leastone processor to: based on the space motion information including theposition information of the target object, update a position of theobject model of the target object in the three-dimensional scene modelaccording to the position information; and based on the space motioninformation including the rotation angle information of the targetobject, update spatial orientation of the object model of the targetobject in the three-dimensional scene model according to the rotationangle information.
 17. The augmented reality display device according toclaim 16, wherein the obtaining code is further configured to cause theat least one processor to: obtain distance data of the target objectaccording to at least two proximity sensors configured to monitor thetarget object, and determining a position information of the targetobject in the target site area after the target object moves; calculate,according to motion data sensed by a positioning sensor disposed on thetarget object, the position of the target object in the target site areaafter the target object moves; and calculate, according to data sensedby a ground auxiliary positioning apparatus disposed in the target sitearea, at least one of the position information of the target object, aposition of a partial structure of the target object in the target sitearea, a height of the target object or a height of the partial structureof the target object falling within a preset height threshold range. 18.The augmented reality display device according to claim 17, wherein theobtaining code is further configured to cause the at least one processorto: obtain sensor position information of the at least two proximitysensors; and performing least square fitting calculation on the obtaineddistance data and sensor position information to obtain the positioninformation of the target object in the target site area.
 19. Theaugmented reality display device according to claim 15, wherein thetarget site area comprises at least two image sensors configured tomonitor the target object from different angles to obtain image data ofthe target object; and wherein the updating code is further configuredto cause the at least one processor to identify an image object of thetarget object from an image acquired by the at least two image sensors,and correcting the updated object model according to the identifiedimage object to obtain a second updated object model of the targetobject.
 20. The augmented reality display device according to claim 15,wherein the determining code is further configured to cause the at leastone processor to: obtain the position information and the field of viewinformation of the display device in the target site area, the displaydevice being configured to display the virtual information of theaugmented reality scene; and determine a user field of view according tothe position information and the field of view information, anddetermine the target area in the established three-dimensional scenemodel according to the user field of view.